Computers and information technology may be quite complex. From the software and middleware to the hardware and connections for communicating between machines, the technologies such as hardware, protocols, languages, software packages, and software standards involved is large and changes rapidly. Accordingly, companies, governments, associations and other organizations rely on proficient programmers, network administrators, database administrators, and other IT (Information Technology) professionals to keep their computer systems functioning properly and efficiently.
To become and remain employable, these IT professionals must be able to keep up and prove they can provide the services the employer seeks, often before being offered an IT employment position. Due to the high costs and high risks of using unskilled employees, employers are often unwilling to allow employees to learn the necessary skills on the job. In addition, universities and colleges do not teach all of the specific programming languages, hardware devices, or protocols used in the industry. Instead, using a select few, they focus on theory and principles common across the technologies.
Thus, to provide the credentials demanded by employers, the IT industry uses certification exams, particularly in the software industry. This effort is often led by vendors of the different technologies. The certification asserts that the vendor considers the certificate holder competent to use and maintain the IT technology being certified. Novell®, Microsoft® and others offer certification exams such as CNE (Certified NetWare Engineer), MCP (Microsoft Certified Professional), MCSE (Microsoft Certified Systems Engineer), and others.
However, conventional certification exams which use multiple choice and fill-in-the-blank questions are limited in their ability to measure behavioral skills. Some examinees may simply be good at memorizing rules and appropriate answers but lack the skills needed to perform “real world” tasks. Alternatively, the examinees may prepare for the exam by storing the answers in short term memory. Once the exam is over and the examinee has passed, the facts are quickly forgotten.
Essentially, the examinee lacks the kind of knowledge which is best obtained through the experience of applying the concepts and principles in real life situations or in a simulated environment. As competition has increased in the IT field, the certification exams have begun to focus greater proportions of their exams on testing an examinee's knowledge and skill in relation to an IT system rather than their ability to retain and recall technical information. This means the exams are setting objectives for the examinees which conventional test questions fail to adequately meet or measure.
For example, rather than require that an examinee know how to configure a network card. The certification exams are requiring that the examinee demonstrate an ability to configure the network card. This kind of objective is best assessed using a simulation or an actual IT environment.
The certification exams of today are designed to test an examinee's competence in using the particular technology, software, administration tool, or the like. These are the skills the employers want to know an applicant possesses. Crafting an exam question to test an examinee's ability to use the technology is very difficult without conducting a one-on-one assessment much like an actual driving test with a DMV official to receive a driver's license. Generally, due to the number of exams given and the number of examinees, one-on-one assessments are impractical.
To address these limitations, interactive exam questions have been developed which are directed toward an examinee's behavior. The examinee must do something to demonstrate that they could perform a specific task in an actual system, if required to do so. These interactive exam questions provide some realism when compared against standard exam questions. For example, one or more screen shots of a tool, configuration window, or other administrative context may be displayed to provide context for the question. The user is then asked to identify which parameter is set incorrectly. The user may identify the parameter by dragging and dropping a correct parameter setting to an answer box.
Alternatively, a conventional simulation may be used. Conventional simulations include a multi-media presentation of images and user interface controls which allow very limited interaction between the user and the simulation. These simulations may provide an interface which allows an examinee to navigate through a simulated software product to the proper location to resolve a problem described in an exam question. However, these simulations only provide user interface functionality for the menus, windows, and controls necessary to navigate to the proper software window to answer the question and to no other parts of the environment. Because the other menus, controls, and windows do not function, an examinee can find the proper screen or problem context quickly by simply using trial and error, at which point answering the question or performing a task may be trivial. Simulations executed by such simulations are often referred to as “closed” or “dumb” simulations because of the limited functionality and navigation paths.
In contrast, an “open” or “intelligent” simulation is one which realistically simulates a computer environment within which a simulation scenario takes place. The open simulation includes logic, rules, and processing ability which allows changes made by a user in the simulation to have a corresponding effect in the simulation. In addition, the open simulation matches the behavior of the actual components of the computer environment so that a user may take multiple navigation paths. Providing multiple navigation paths and ways to perform a task allows a user to truly “trouble shoot” a problem as will be expected in the actual computer environment. The user is free to investigate the simulated computer environment to solve a problem in a manner similar to real life.
Unfortunately, conventional simulators which provide limited openness in the simulations have other limitations. Generally, the data defining a simulation and/or computer environment is tightly integrated with the executable code of the simulators. Therefore, development and revisions to simulations is very expensive because this must be done by programmers instead of less expensive test item writers. The programmers modify the actual simulators to produce different simulations.
One element which quickly defeats the realism of a simulation, is how closely time delays of an actual computer system are simulated. Ideally, for each action performed in a simulation of a computer environment, the delay should be about the same as the delay in an actual computer environment. Users, who interact daily with computers, quickly notice unnatural delays in a simulated computer environment from a few seconds to even less than a second. Retrieving and modifying data from files or a database for use within a simulation can add unavoidable delays in responses to user actions in the simulation. These delays can annoy and distract a user.
Data for a simulation may be stored in data files on the same machine executing the simulation software. Retrieving data from the files may incur a significant time delay because, on average, at least half of the file must be searched for each data value request. A single file or a plurality of separate files may be used.
If a single file is used, the file size may grow so large that reading and re-reading the file to access data causes unnatural time delays in the simulation. If multiple files are used, the files are typically ‘flat,’ meaning the data is stored in a list of records and the files are not related to each other. Retrieving data may require searching multiple files. The searching, maintaining, and distributing of the data files may be time consuming, complex, and error prone.
To overcome these problems, simulations may utilize a database. Conventional databases are DBMS (Database Management Systems) that provide a very flexible and powerful database which may include features such as automatic backups, security, type checking, integrity constraints and the like. However, in relation to simulations, generally, the more flexible and feature laden a DMBS is, the more overhead processing is involved, causing a greater delay when operating on the data.
Generally, simulation software executes a data request that is serviced by a database communication layer, a database driver, associated with the database. The database driver is software that controls access to the database. The data request is generally in a standardized format such as a SQL (Structured Query Language) statement. Once a data request is made, the SQL statement is interpreted and executed to provide a response that includes the desired data. The extra processing time required for SQL statements may cause noticeable delays in the behavior of the simulation.
The processing time may be further delayed if a RDBMS (Relational Database Management Systems) is used. In a RDBMS, data is organized according to relationships between different sets of data. Optimally, the relationships model real-world relationships between entities that the data is intended to define and represent. When the relationship matches a real-world relationship, the data may be stored in a single table or record of the database. Consequently, database requests involving the data have a minimal time delay.
However, certain data requests involve data which is not related in a manner directly represented in the database. To fulfill such data requests, multiple data indexes, tables, and records may be searched and accessed. This overhead increases the delay in the simulation requesting the data.
DBMSes comprise data files which store data and at least one database driver which is computer-executable code for controlling how the data is stored, accessed, and maintained. The database drivers are separate from the data files and software programs, such as a simulator which uses the database. If the DBMS operates as a server, database drivers may be required on the client computers as well as the server computer.
As a DBMS is upgraded or repaired to resolve software bugs, one or more of these database drivers may need to be replaced. Maintaining the proper versions of database drivers may be burdensome, particularly in a computer lab used to administer a certification exam which includes simulations relying on the database drivers for data from the database. The computer lab may include fifty to one-hundred or more computers that must have the most current database drivers.
A DBMS may involve separate software programs for installation and maintenance of the database. Depending on the size and configuration of the database, some DBMSes require a full-time staff person to administer and maintain the database. DBMSes are very flexible such that data may be provided to a variety of different software applications. However, the flexibility increases delays involved in completing database operations.
To edit data of conventional simulations, the simulation source code is edited and recompiled. A DBMS provides an interface for editing the data. However, edits are generally accomplished using SQL statements. To properly perform edits of the data, users must be trained in the SQL language. This complexity increases the costs of developing and maintaining simulations.